Method and apparatus for constructing sensory effect media data file, method and apparatus for playing sensory effect media data file, and structure of the sensory effect media data file

ABSTRACT

Disclosed is a method and apparatus for constructing and playing a sensory effect media data file, which includes information on sensory effects. A method for constructing a sensory effect media data file according to an embodiment of the present invention includes: inserting first composition information, which represents a property of media data, into a composition information container field; inserting second composition information, which represents a property of sensory effect data, into the composition information container field; inserting a sample of the media data into a media data field, where the sample represents data associated with a timestamp; and inserting a sample of the sensory effect data into the media data field.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Phase Application of PCT InternationalApplication No. PCT/KR2014/000045, which was filed on Jan. 3, 2014, andwhich claims priority from Korean Patent Application No.10-2013-0124897, filed with the Korean Intellectual Property Office onOct. 18, 2013, and Korean Patent Application No. 10-2013-0154236, filedwith the Korean Intellectual Property Office on Dec. 11, 2013. Thedisclosures of the above patent applications are incorporated herein byreference in their entirety.

BACKGROUND

1. Technical Field

The present invention relates to a method and apparatus for constructingand playing media data, more particularly to a method and apparatus forconstructing and playing a sensory effect media data file that includessensory effect information.

2. Description of the Related Art

The typical format of a media file may basically include a headerportion, which describes the information of the corresponding mediacontent, and a video data portion, which stores the media data in acompressed form. While the typical format of a media file describedabove can be used to store simple video data, it may not be suitable asa comprehensive structure for holding various types of media.

Accordingly, the international standards organization MPEG defined theISO base media file format as a basic file format that can be commonlyapplied to various applications. The ISO base media file format isdesigned to store data, such as a compressed media stream, compositioninformation related to the media stream, and the like, in multiplecontainers in a hierarchical manner. The ISO base media file format doesnot define encryption and decryption methods but rather defines a basicstructure for efficiently storing encrypted or decrypted media streams.

As can be seen from HD broadcasting and 3D broadcasting, not to mentionIMAX and 4D movies, there is a growing interest in methods for providingmore realistic videos, in industries related to video contents. In orderto allow the user to become immersed in a video content and feel as ifhe/she is one of the characters in the story, it is necessary not onlyto provide visual and auditory engrossment but also to maximizeengrossment via stimulations of other senses such as the olfactory andtactile senses. To do so, it is necessary not only to provide higherlevels of visual engrossment, such as by the ultrahigh resolutions and3D displays, and higher levels of auditory realism, such as by 3D audiodisplays, as for existing video contents, but also to apply adjustmentsto sensory effect metadata (SEM), such as lighting, wind, temperature,vibrations, movement, tactile feels, scents, etc., in a manner suitableto the user's viewing environment and thereby maximize the engrossmentwith the video content.

Broadcasts or video contents that stimulate the user's five senses bythus utilizing sensory effect devices in the user's surroundings arereferred to as sensory contents or 4D media.

Existing 4D media using SEM is difficult to create, since theaudiovisual content data and the SEM are separated, and it is thusdifficult to effectively store, exchange, manage, and play the media.Prior art related to this subject includes Korean Patent Publication No.10-2012-0106157.

Thus, there is a need for more research regarding methods foreffectively storing, exchanging, managing, and playing 4D media.

SUMMARY

An objective of the invention is to provide a method and apparatus forconstructing a file by integrating sensory effect data with media dataand provide a method and apparatus for playing the sensory effect mediadata file formed by the sensory effect data and the media dataintegrated together.

An objective of the invention is to provide a method and apparatus forconstructing a sensory effect media data file by dividing the sensoryeffect data into samples and inserting a divided sample of the sensoryeffect data and a sample of the media data into the same field, and toprovide a method and apparatus for playing such sensory effect mediadata file.

To achieve the objectives above, an embodiment of the invention providesa method for constructing a sensory effect media data file thatincludes: inserting first composition information, which represents aproperty of media data, into a composition information container field;inserting second composition information, which represents a property ofsensory effect data, into the composition information container field;inserting a sample of the media data into a media data field, where thesample represents data associated with a timestamp; and inserting asample of the sensory effect data into the media data field.

To achieve the objectives above, an embodiment of the invention providesan apparatus for constructing a sensory effect media data file thatincludes: a composition information constructor unit configured toinsert first composition information and second composition informationinto a composition information container field, where the firstcomposition information represents a property of media data, and thesecond composition information represents a property of sensory effectdata; a sample constructor unit configured to insert a sample of themedia data into a media data field and insert a sample of the sensoryeffect data into the media data field, where the sample represents dataassociated with a timestamp; and a file constructor unit configured toconstruct a sensory effect media data file by combining the firstcomposition information, the second composition information, the sampleof the media data, and the sample of the sensory effect data.

To achieve the objectives above, an embodiment of the invention providesa method for playing a sensory effect media data file that includes:separating a composition information container field and a media datafield included in a sensory effect media data file; detecting firstcomposition information and second composition information by parsingthe separated composition information container field, where the firstcomposition information represents a property of media data, and thesecond composition information represents a property of sensory effectdata; detecting a sample of the media data and a sample of the sensoryeffect data by parsing the separated media data field, where the samplerepresents data associated with a timestamp; and playing the sensoryeffect media data by using the first composition information, the secondcomposition information, the sample of the media data, and the sample ofthe sensory effect data.

To achieve the objectives above, an embodiment of the invention providesan apparatus for playing a sensory effect media data file that includes:a file separator unit configured to separate a composition informationcontainer field and a media data field included in a sensory effectmedia data file; a composition information parser unit configured toparse the separated composition information container field to detectfirst composition information and second composition information, wherethe first composition information represents a property of media data,and the second composition information represents a property of sensoryeffect data; a sample parser unit configured to parse the separatedmedia data field to detect a sample of the media data and a sample ofthe sensory effect data, where the sample represents data associatedwith a timestamp; and a player unit configured to play the sensoryeffect media data by using the first composition information, the secondcomposition information, the sample of the media data, and the sample ofthe sensory effect data.

To achieve the objectives above, an embodiment of the invention providesa sensory effect media data file structure that includes: a compositioninformation container field in which first composition information andsecond composition information are inserted, where the first compositioninformation represents a property of media data, and the secondcomposition information represents a property of sensory effect data;and a media data field in which a sample of the media data and a sampleof the sensory effect data are inserted, where the sample representsdata associated with a timestamp.

To achieve the objectives above, an embodiment of the invention providesa sensory effect media data file structure that includes: a compositioninformation container field in which composition information isinserted, where the composition information represents a property ofmedia data; a media data field in which a sample of the media data isinserted, where the sample represents data associated with a timestamp;and a meta field in which sensory effect metadata is inserted.

According to an embodiment of the invention, the sensory effect data andmedia data can be integrated to form and be processed as a single file,thereby making it possible to effectively store, manage, and play thesensory effect media data.

According to an embodiment of the invention, the sensory effect data canbe divided into samples, and the divided sample of the sensory effectdata and a sample of the media data can be inserted into the same fieldto construct a sensory effect media data file, to allow usefulapplication to streaming services or broadcasting services.

Additional aspects and advantages of the present invention will be setforth in part in the description which follows, and in part will beobvious from the description, or may be learned by practice of theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an apparatus for constructing a sensoryeffect media data file related to an embodiment of the invention.

FIG. 2 and FIG. 3 are diagrams illustrating the structure of a sensoryeffect media data file related to an embodiment of the invention.

FIG. 4 is a diagram for describing a method of dividing sensory effectdata into timed data and untimed data related to an embodiment of theinvention.

FIG. 5 illustrates a method of constructing timed data from amongsensory effect data related to an embodiment of the invention.

FIG. 6 illustrates a method of constructing group effect data forexpressing multiple sensory effects related to an embodiment of theinvention.

FIG. 7 illustrates a method of constructing untimed data from amongsensory effect data related to an embodiment of the invention.

FIG. 8 is a diagram for describing a method of constructing thetransmission stream sample of a sensory effect media data file relatedto an embodiment of the invention.

FIG. 9 is a block diagram of an apparatus for playing a sensory effectmedia data file related to an embodiment of the invention.

DETAILED DESCRIPTION

A method and apparatus for constructing a sensory effect media datafile, a method and apparatus for playing a sensory effect media datafile, and a structure for the sensory effect media data file will bedescribed below in more detail with reference to the accompanyingdrawings.

In the present specification, an expression used in the singularencompasses the expression of the plural, unless it has a clearlydifferent meaning in the context. In the present specification, termssuch as “comprising” or “including,” etc., should not be interpreted asmeaning that all of the elements or operations are necessarily included.That is, some of the elements or operations may not be included, whileother additional elements or operations may be further included.

FIG. 1 is a block diagram of an apparatus for constructing a sensoryeffect media data file related to an embodiment of the invention.

As illustrated in FIG. 1, a sensory effect media data file constructionapparatus 100 can include a composition information constructor unit110, a sample constructor unit 120, and a file constructor unit 130.

In the present specification, media data can include video data, voicedata, text data, etc., and can be a combination of at least one or twoor more of video data, voice data, and text data. Also, the video datacan include three-dimensional video data such as stereoscopic images.

In the present specification, sensory effect data (sensory effectmetadata, SEM) may be data that represents sensory effect information,while the sensory effect information may be information that can providethe user of the media data with a stimulation of the visual, auditory,tactile sensation, etc., and may be information that can express light,flashes, heating, cooling, wind, vibration, scents, fog, spraying, colorcorrection, tactile sensation, kinesthetic sensation, rigid body motion,etc. The sensory effect data can take the form of an xml instance.

The sensory effect data can be constructed as DescriptionMetadata,Declarations, GroupOfEffects, Effect, or ReferenceEffect.

DescriptionMetadata, based on Type, may be used when describingauthorship information such as the last update time, comments, creators,location of creation, etc., of a sensory effect, while Declarations,GroupOfEffects, Effect, and ReferenceEffect may be used when expressingthe sensory effect. Declarations may be used to pre-define sensoryeffects that are frequently used. ReferenceEffect may be used whenreferencing sensory effects that were defined in an external resource orsensory effects that were defined in Declarations, or when reusingsensory effects that have been used already.

Effect may be used when expressing one sensory effect, whileGroupOfEffects may be used when expressing two or more sensory effects.

Also, a sensory effect media data file refers to a file constructed withthe media data and the sensory effect data integrated together.

The media data can be inputted to the composition informationconstructor unit 110 and the sample constructor unit 120. The media datacan be inputted from the outside.

The sensory effect data can be inputted to the composition informationconstructor unit 110 and the sample constructor unit 120. The sensoryeffect data can be inputted from the outside.

The composition information constructor unit 110 can detect theinformation on a media object forming the media data and can constructthe composition information for each media object (referred to herein as‘first composition information’ for convenience). The first compositioninformation can represent a property of the media data. For example, thecomposition information constructor unit 110 can construct thecomposition information to include information on the size of the videodata included in the media data, information for defining the type ofthe encryption stream of the media data, information related to thecamera used for obtaining the video, display information required fordisplaying the video, information on the frame rate of the video date,the number of field lines for frames forming the video data, etc.

Furthermore, in cases where the media data includes a three-dimensionalvideo, the composition information constructor unit 110 can furtherinclude disparity information between the left video and right videoincluded for the three-dimensional video. Also, the compositioninformation can include address information for samples of the mediadata.

In the present specification, a sample can mean any data associated witha timestamp.

Also, the composition information constructor unit 110 can constructcomposition information representing a property of sensory effect data(referred to herein as ‘second composition information’ forconvenience). For example, the second composition information caninclude description information on a sensory effect data sample andaddress information for the sensory effect data sample.

The composition information constructor unit 110 can insert the firstcomposition information and the second composition information into thesame field (for instance, the composition information container field).

Also, the sample constructor unit 120 can construct the sample of themedia data and the sample of the sensory effect data and can insert theminto the media data field. The media data sample can be encrypted data.Also, the sample of the sensory effect data can be timed data from amongthe sensory effect data.

According to an embodiment of the present invention, the sensory effectdata can be divided into timed data and untimed data. Timed data refersto data that is affected by time, while untimed data refers to data thatis not affected by time.

According to another embodiment of the present invention, the sensoryeffect media data file can be constructed without forming the sensoryeffect data as samples but by inserting the xml instance form as is intothe meta box.

In the embodiments below, the sensory effect media data file may bebased on the ISO base media file format to allow combinations of variouscomponent standards. File formats based on the ISO base media fileformat include the MPEG-4 file format (ISO/IEC 14496-14), MPEG-21 fileformat (ISO/IEC 21000-9), etc. Such a file format is resilient, has verygood expandability, and is advantageous for streaming services.

The ISO base media file format is basically composed of sets calledboxes. The type of a box is indicated by a unique identifier made as anabbreviation of four letters. For example, the box type of the mediadata box is ‘mdat’.

FIG. 2 shows the structure of a sensory effect media data fileconstructed by using a meta box.

The illustrated sensory effect media data file can include the ftyp box210, moov box 220, mdat box 230, and meta box 240. In the ftyp box 210,information on the file type and information on compatibility can beinserted. The moov box 220 may be the composition information containerfield, and can have the composition information of the media datainserted therein. For example, the moov box 220 can include track boxesfor the media data (track for video, track for audio), and the addressinformation of the media data sample can be stored in the track boxesfor the media data. The mdat box 230 may be the media data field, andcan have the samples of the media data (Video Sample, Audio Sample)inserted therein. Also, the meta box 240 may be the container box of thexml box.

The meta box 240 can include an xml box that carries the xml. Theembodiment shown in FIG. 2 is possible when considering only simplestorage. The embodiment above entails the advantage of easyimplementation.

As another embodiment of the present invention, the sensory effect mediadata file can be constructed using track boxes.

FIG. 3 shows the structure of a sensory effect media data fileconstructed by using track boxes.

The illustrated sensory effect media data file can include the ftyp box310, moov box 320, and the mdat box 330. The ftyp box 310 can have theinformation on the file type and information on compatibility insertedtherein. The moov box 320 may be the composition information containerfield, and can have the composition information of the media datainserted therein. For example, the moov box 320 can include track boxesfor the media data (track for video, track for audio), and a track boxfor the sensory effect data (track for SEM). The address information ofthe media data sample can be stored in the track boxes for the mediadata, and description information and address information of the sensoryeffect data sample can be included in the track box for the sensoryeffect data. The mdat box 330 may be the media data field, and can havethe samples of the media data (Video Sample, Audio Sample) and samplesof the sensory effect data (Sensory Effect Metadata Sample) insertedtherein.

That is, instead of carrying the sample directly, the track may take thestructure of referencing the sample held in the mdat box. A major issuein the scheme of referencing a SEM sample held in the mdat box by way ofa SEM track is to divide the SEM xml instance according to certaincriteria and storing in the mdat, in order to create samples from theSEM. With this embodiment, the sensory effect data (SEM) may be storedin a divided form, and hence can be suitable for a streaming service ora broadcast service environment that employs MPEG-2 TS using hinttracks. According to this embodiment, when a TV is turned on for viewingsensory effect media data, the SEM samples corresponding to the currentpresentation time may be downloaded, instead of receiving the entireSEM. Thus, the user does not have to wait for the downloading of theentire SEM. Also, in terms of the performance of a sensory device, itmay be sufficient to decode only the information regarding the effectscorresponding to the current presentation time, so that there is lessdependence on performance.

In order to construct such a sensory effect media data file that usesSEM tracks, the SEM may be divided into timed data (i.e. metadata thatis affected by time; timed metadata) and untimed data (metadata that isnot affected by time; untimed metadata).

FIG. 4 is a diagram for describing a method of dividing sensory effectdata into timed data and untimed data related to an embodiment of theinvention.

FIG. 4 uses an XML instance of SEM as an example to show the results ofdividing into timed data and untimed data. From among the SEM, the timeddata is GroupOfEffects, Effect, and ReferenceEffet and correspond tosamples of mdat, while the untimed data is DescriptionMetadata andDeclarations and is included in the stsd and sgpd boxes.

The timed data from among the sensory effect data (SEM) is entered intothe mdat box. From among the timed data, Effect and ReferenceEffect eachform a sample. These samples may be stored in the mdat box, and thelocations of the samples in mdat can be found from the chunk_offsetvalues of the stco box.

FIG. 5 illustrates a method of constructing timed data from amongsensory effect data related to an embodiment of the invention.

FIG. 5 shows how the chunk_offset of the stco box (the box in whichoffset information is inserted) points to SEM samples within mdat. Thesingle effect and single reference effect can find the offsets of theSEM samples within mdat through the stco box. On the other hand, GroupOf Effects may subdivide the effects and reference effects therein suchthat each forms a SEM sample. However, after such subdivision in themanner described above, it may not be known which SEM sample belonged towhich Group Of Effects. If the original SEM XML instance can berecovered at the sensory effect media data file, it is possible to editthe recovered SEM XML instance by using an existing SEM authoring tooland recreate the sensory effect media data file.

FIG. 6 illustrates a method of constructing group effect data forexpressing multiple sensory effects related to an embodiment of theinvention. That is, a method of constructing a file box according toGroupOfEffects is illustrated.

As illustrated in the drawings, the sbgp box may be the group samplefield, and SEM samples can be grouped by way of the group descriptionindex and sample count information (sample_count) of the sbgp box.Regarding the contents of the sbgp box, a group description index valueof 0 may represent a single effect or a single reference effect (aneffect that is not included in Group Of Effects), while a groupdescription index of 1 or higher may represent effects or referenceeffects within a Group Of Effects. As the number of the Group Of Effectsis increased, so also does the group description index value increase.

Also, the sample count may represent the number of consecutive samplesthat have the same group description index value. In FIG. 6, the firsteffect is a single effect.

Thus, the group description index is 0, and since the consecutive numberof single effects is one, the sample count is 1. The second, third, andfourth effects are effects that belong to GroupOfEffect. Since this isthe first GroupOfEffects, the group description index is 1, and sincethe number of consecutive effects of this GroupOfEffects is three, thesample count is 3. After subdividing the GroupOfEffects by this methodto provide single effect samples, these can be grouped together. Also,it is also possible to recover the GroupOfEffects from the subdividedsingle effect samples.

The untimed data from among the sensory effect data can be inserted intothe stsd box (the box in which sample description information isinserted) and the sgpd box (the box in which sample group descriptioninformation is inserted) of the SEM track, which is a different fieldfrom the media data field.

FIG. 7 illustrates a method of constructing untimed data from amongsensory effect data related to an embodiment of the invention.

FIG. 7 shows how untimed data may be held in the stsd box and sgpd box.The boxes can be newly defined in table form as Table 1 and Table 2.

TABLE 1 // MPEG-V Part3 Sample Entry aligned(8) classMPEGVPart3SampleEntry extends MetaDataSampleEntry(‘mvp3’ { /* xml orbinary flag */ unsigned int(8) binaryFlag; // 0: binary, 1: xml /* anyattribute */ unsigned int(8) isAnchorElement; if(isAnchorElement == 1) {unsigned int(8) anchorElement; } unsigned int(8) isEncodeAsRAP;if(isEncodeAsRAP == 1) { unsigned int(8) encodeAsRAP; }   unsignedint(8) isPuMode; if(isPuMode == 1) { string puMode; }   unsigned int(8)isTimeScale; if(isTimeScale == 1) { unsigned int(32) timeScale; }  unsigned int(8) isPtsDelta; if(isPtsDelta == 1) { unsigned int(32)ptsDelta; }   unsigned int(8) isAbsTimeScheme; if(isAbsTimeScheme == 1){ string absTimeScheme; }   unsigned int(8) isAbsTime; if(isAbsTime== 1) { string absTime; }   unsigned int(8) isPts; if(isPts == 1) {unsigned int(32) pts; }   unsigned int(8) isAutoExtractionType;if(isAutoExtractionType == 1) { string autoExtractionType; } /*DescriptionMetadata, Declaration element */ unsigned int(8)isDescriptionMetaData; if(isDescriptionMetaData == 1) { unsigned int(32)descriptionmetadata_length; unsigned int(8) descriptionmetadata[ ]; }unsigned int(8) isDeclaration; if(isDeclaration == 1) { unsigned int(32)declaration_count; unsigned int(32) declaration_length[ ]; unsignedint(8) declaration[ ][ ]; } }

The MPEGVPart3SampleEntry('mvp3′) of Table 1 is a newly defined box andinherited MetaDataSampleEntry defined in the ISO Base Media File Format.This is located inside the stsd box. The contents held inMPEGVPart3SampleEntry include a Binary flag, attributes of SEM rootelements, Description Metadata elements, and Declarations elements. TheBinary flag may determine whether the timed SEM data and untimed SEMdata are to be stored in string form or binary form. The rules whenstoring in binary form may follow the binary representation syntaxdescribed in MPEG-V Part3 (ISO/IEC 23005-3). The attributes of SEM rootelements may include information such as time scale, pts, pts delta,etc. The Description Metadata elements may be used to hold authorshipinformation, and the Declarations element may be used to pre-defineeffects that are frequently used.

TABLE 2 // Group Of Effects Description Entry aligned(8) classGroupOfEffectsDescriptionEntry extendsSampleGroupDescriptionEntry(‘goed’) { /* any attribute */ unsignedint(8) isAnchorElement; if(isAnchorElement == 1) { unsigned int(8)anchorElement; } unsigned int(8) isEncodeAsRAP; if(isEncodeAsRAP == 1) {unsigned int(8) encodeAsRAP; } unsigned int(8) isPuMode; if(isPuMode== 1) { string puMode; } unsigned int(8) isTimeScale; if(isTimeScale== 1) { unsigned int(32) timeScale; } unsigned int(8) isPtsDelta;if(isPtsDelta == 1) { unsigned int(32) ptsDelta; } unsigned int(8)isAbsTimeScheme; if(isAbsTimeScheme == 1) { string absTimeScheme; }unsigned int(8) isAbsTime; if(isAbsTime == 1) { string absTime; }unsigned int(8) isPts; if(isPts == 1) { unsigned int(32) pts; } unsignedint(8) isAutoExtractionType; if(isAutoExtractionType == 1) { stringautoExtractionType; } /* sem base attributes */ unsigned int(8)isActivate; if(isActivate == 1) { unsigned int(8) activate; } unsignedint(8) isDuration; if(isDuration == 1) { unsigned int(32) duration; }unsigned int(8) isFade; if(isFade == 1) { unsigned int(32) fade; }unsigned int(8) isAlt; if(isAlt == 1) { string alt; } unsigned int(8)isPriority; if(isPriority == 1) { unsigned int(32) priority; } unsignedint(8) isLocation; if(isLocation == 1) { string location; } /* semadaptability attribute */ unsigned int(8) isAdaptType; if(isAdaptType== 1) { string adaptType; } unsigned int(8) isAdaptRange;if(isAdaptRange == 1) { string adaptRange; } }

The GroupOfEffectsDescriptionEntry of Table 2 is a newly defined box andinherited SampleGroupDescriptionEntry defined in the ISO Base Media FileFormat. This is located inside the sgpd box. The attributes ofGroupOfEffects may be held in GroupOfEffectsDescriptionEntry.

Other advantages of constructing sensory effect media data with the SEMtracks described above are the efficient storing and transmitting ofMPEG-2 TS packets.

FIG. 8 is a diagram for describing a method of constructing thetransmission stream sample of a sensory effect media data file relatedto an embodiment of the invention.

FIG. 8 shows a method of constructing a TS (transport stream) sample byusing a Hint track. The hint sample of the mdat box represents one TSpacket. A TS packet is composed of a header and a payload. The headerincludes information such as the sync byte for synchronization with adecoder, PID, which is the only value for differentiating packets, andthe like. The information of the header includes information fortransmissions which are unrelated to audiovisual contents. In contrast,the payload may be composed of audio, video, and other media. The Hinttrack may aid the construction of the information carried in the payloadof the TS packet by referencing the video sample and audio sample. Inthis configuration, the offsets of the media samples (video samples,audio samples, and other media samples) are carried by the payloads ofthe TS samples. If the Hint tracks are not used, separate MAF files forplaying and separate TS files for transmission may be required. By usingHint tracks, however, one file can fulfill the purposes of both playingand transmission, and since the payload of a TS sample carries only theoffset of a sample, the storage capacity can be reduced. This is animportant reason for creating a sensory effect media data file by usingSEM tracks. If hint tracks are used in a sensory effect media data filethat employs SEM tracks, one file can satisfy the purposes of playingand transmitting 4D Media (or sensory media), and this advantage canprovide a foundation by which not only large-scale broadcastingcompanies but also individual users may utilize 4D broadcasts easily andconveniently.

FIG. 9 is a block diagram of an apparatus for playing a sensory effectmedia data file related to an embodiment of the invention. With thisembodiment, a method of for playing a sensory effect media data file byusing a track box will be described.

As illustrated in the drawing, a sensory effect media data file playerapparatus 900 can include a file separator unit 910, a compositioninformation parser unit 920, a sample parser unit 930, and a player unit940.

The file separator unit 910 can receive a sensory effect media data fileas input from the outside, and can separate the composition informationcontainer field and the media data field included in the sensory effectmedia data file.

The information included in the composition information container fieldcan be inputted to the composition information parser unit 920, and theinformation included in the media data field can be inputted to thesample parser unit 930.

The composition information parser unit 920 may parse the separatedcomposition information container field to detect the first compositioninformation, which represents the properties of the media data, and thesecond composition information, which represents the properties of thesensory effect data.

Also, the sample parser unit 930 can parse the separated media datafield to detect samples of the media data and samples of the sensoryeffect data.

The composition information of the media data, the compositioninformation of the sensory effect data, the media data samples, and thesensory effect data samples that are detected from the parsing by thecomposition information parser unit 920 and the parsing by the sampleparser unit 930 can be inputted to the player unit 940.

The player unit 940 can combine the composition information of the mediadata, composition information of the sensory effect data, media datasamples, and sensory effect data samples to play the sensory effectmedia data file.

According to an embodiment of the present invention, the sensory effectmedia data file player apparatus 900 can also play a sensory effectmedia data file constructed using meta boxes as in FIG. 2. In this case,the composition information of the media data, the media data samples,and the sensory effect data in the form of xml instances, as included inthe sensory effect media data file can be separated, and the sensoryeffect media data file can be played by using the compositioninformation of the media data, media data samples, and sensory effectdata in the form of xml instances.

According to an embodiment of the present invention, sensory effect dataand media data can be integrated into a single file for processing, sothat the sensory effect media data can be stored, managed, and playedeffectively.

According to an embodiment of the present invention, sensory effect datacan be divided into samples, and the samples of the sensory effect dataand samples of media data can be inserted in the same field inconstructing a sensory effect media data file, thereby allowing usefulapplication to streaming services or broadcasting services.

The method of constructing a sensory effect media data file and themethod of playing a sensory effect media data file described above canbe implemented in the form of program instructions that may be performedusing various computer means and can be recorded in a computer-readablemedium. Such a computer-readable medium can include programinstructions, data files, data structures, etc., alone or incombination. The program instructions recorded on the medium can bedesigned and configured specifically for the present invention or can bea type of medium known to and used by the skilled person in the field ofcomputer software.

Examples of a computer-readable medium may include magnetic media suchas hard disks, floppy disks, magnetic tapes, etc., optical media such asCD-ROM's, DVD's, etc., magneto-optical media such as floptical disks,etc., and hardware devices such as ROM, RAM, flash memory, etc.

Examples of a computer-readable medium can also include a transmittingmedium such as light, metal lines, waveguides, etc., that transmitssignals for specifying program instructions, data structures, etc.

Examples of the program of instructions may include not only machinelanguage codes produced by a compiler but also high-level language codesthat can be executed by a computer through the use of an interpreter,etc. The hardware mentioned above can be made to operate as one or moresoftware modules that perform the actions of the embodiments of theinvention, and vice versa.

The method and apparatus for constructing a sensory effect media datafile and the method and apparatus for playing a sensory effect mediadata file described above are not to be limited in their application tothe compositions and methods of the embodiments described above. Rather,some or all of each of the embodiments may be selectively combined toform numerous variations.

DESCRIPTION OF NUMERALS

100: sensory effect media data file construction apparatus

110: composition information constructor unit

120: sample constructor unit

130: file constructor unit

900: sensory effect media data file player apparatus

910: file separator unit

920: composition information parser unit

930: sample parser unit

940: player unit

What is claimed is:
 1. A method for constructing a sensory effect mediadata file, the method comprising: inserting first compositioninformation into a composition information container field, the firstcomposition information representing a property of media data; insertingsecond composition information into the composition informationcontainer field, the second composition information representing aproperty of sensory effect data; inserting a sample of the media datainto a media data field, the sample representing data associated with atimestamp; and inserting a sample of the sensory effect data into themedia data field.
 2. The method of claim 1, wherein the sample of thesensory effect data inserted into the media data field comprises timeddata from among the sensory effect data, the timed data being dataaffected by time.
 3. The method of claim 2, wherein the timed datacomprises group effect data for expressing at least two sensory effects,and the group effect data comprises samples of data for the at least twoof sensory effects.
 4. The method of claim 3, further comprising:inserting a group description index and sample count information into agroup sample field, the group description index representingidentification information of a group, the sample count informationrepresenting a number of consecutive samples having an identical groupdescription index.
 5. The method of claim 3, further comprising:inserting untimed data from among the sensory effect data into a fielddifferent from the media data field, wherein the untimed data is dataunaffected by time.
 6. An apparatus for constructing a sensory effectmedia data file, the apparatus comprising: a composition informationconstructor unit configured to insert first composition information andsecond composition information into a composition information containerfield, the first composition information representing a property ofmedia data, the second composition information representing a propertyof sensory effect data; a sample constructor unit configured to insert asample of the media data into a media data field and insert a sample ofthe sensory effect data into the media data field, the samplerepresenting data associated with a timestamp; and a file constructorunit configured to construct a sensory effect media data file bycombining the first composition information, the second compositioninformation, the sample of the media data, and the sample of the sensoryeffect data.
 7. The apparatus of claim 6, wherein the sample of thesensory effect data inserted into the media data field comprises timeddata from among the sensory effect data, the timed data being dataaffected by time.
 8. The apparatus of claim 7, wherein the timed datacomprises group effect data for expressing at least two sensory effects,and the group effect data comprises samples of data for the at least twosensory effects.
 9. The apparatus of claim 8, wherein the compositioninformation constructor unit inserts a group description index andsample count information into a group sample field, the groupdescription index representing identification information of a group,the sample count information representing a number of consecutivesamples having an identical group description index.
 10. The apparatusof claim 8, wherein the composition information constructor unit insertsuntimed data from among the sensory effect data into a field differentfrom the media data field, wherein the untimed data is data unaffectedby time.
 11. A method for playing a sensory effect media data file, themethod comprising: separating a composition information container fieldand a media data field included in a sensory effect media data file;detecting first composition information and second compositioninformation by parsing the separated composition information containerfield, the first composition information representing a property ofmedia data, the second composition information representing a propertyof sensory effect data; detecting a sample of the media data and asample of the sensory effect data by parsing the separated media datafield, the sample representing data associated with a timestamp; andplaying the sensory effect media data by using the first compositioninformation, the second composition information, the sample of the mediadata, and the sample of the sensory effect data.
 12. The method of claim11, wherein the sample of the sensory effect data inserted in the mediadata field comprises timed data from among the sensory effect data, thetimed data being data affected by time.
 13. The method of claim 12,wherein the timed data comprises group effect data for expressing atleast two sensory effects, and the group effect data comprises samplesof data for the at least two of sensory effects.
 14. The method of claim13, wherein the sensory effect media data file further comprises a groupsample field having a group description index and sample countinformation inserted therein, the group description index representingidentification information of a group, the sample count informationrepresenting a number of consecutive samples having an identical groupdescription index, the separating comprises further separating the groupsample field included in the sensory effect media data file, thedetecting of the first and second composition information comprisesdetecting the group description index and the sample count informationby parsing the group sample field, and the playing comprises furtherusing the group description index and the sample count information. 15.An apparatus for playing a sensory effect media data file, the apparatuscomprising: a file separator unit configured to separate a compositioninformation container field and a media data field included in a sensoryeffect media data file; a composition information parser unit configuredto parse the separated composition information container field to detectfirst composition information and second composition information, thefirst composition information representing a property of media data, thesecond composition information representing a property of sensory effectdata; a sample parser unit configured to parse the separated media datafield to detect a sample of the media data and a sample of the sensoryeffect data, the sample representing data associated with a timestamp;and a player unit configured to play the sensory effect media data byusing the first composition information, the second compositioninformation, the sample of the media data, and the sample of the sensoryeffect data.
 16. The apparatus of claim 15, wherein the sample of thesensory effect data inserted in the media data field comprises timeddata from among the sensory effect data, the timed data being dataaffected by time.
 17. The apparatus of claim 16, wherein the timed datacomprises group effect data for expressing at least two sensory effects,and the group effect data comprises samples of data for the at least twoof sensory effects.
 18. A sensory effect media data file structurecomprising: a composition information container field having firstcomposition information and second composition information insertedtherein, the first composition information representing a property ofmedia data, the second composition information representing a propertyof sensory effect data; and a media data field having a sample of themedia data and a sample of the sensory effect data inserted therein, thesample representing data associated with a timestamp.
 19. The sensoryeffect media data file structure of claim 18, wherein the sample of thesensory effect data inserted in the media data field comprises timeddata from among the sensory effect data, the timed data being dataaffected by time.
 20. A sensory effect media data file structurecomprising: a composition information container field having compositioninformation inserted therein, the composition information representing aproperty of media data; a media data field having a sample of the mediadata inserted therein, the sample representing data associated with atimestamp; and a meta field having sensory effect metadata insertedtherein.